United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/775.490 



02/09/2004 



22801 7590 08/( 

LEE & HAYES, PLLC 

601 W. RIVERSIDE AVENUE 

SUITE 1400 

SPOKANE, WA 99201 



Patrick N.Nelson 



JONES, HEATHER RAE 



PAPER NUMBER 



NOTIFICATION DATE | DELIVERY MODE 
08/04/2010 ELECTRONIC 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 
following e-mail address(es): 

Ihploms (oUeehaves.com 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/775,490 


Applicant(s) 

NELSON, PATRICK N. 


Examiner 

HEATHER R. JONES 


Art Unit 

2621 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 18 May 2010 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-4,7-48 and 50-59 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) |EI Claim(s) 1-4, 7-48 and 50-59 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 09 February 2004 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

El Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date 5/18/2010 . 6) □ Other: . 

PTOL-326 (Rev. 08-06) Office Action Summary Part of Paper No./Mail Date 20100731 



Application/Control Number: 10/775,490 Page 2 

Art Unit: 2621 

DETAILED ACTION 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1-4, 7-48, and 50-59 have 
been considered but are moot in view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a patent 
therefor, subject to the conditions and requirements of this title. 

Claims 42-48, 50, and 51 are rejected under 35 U.S.C. 101 because the 
claimed invention is directed to non-statutory subject matter. Claims 42-48, 50, 
and 51 define a computer-readable storage medium embodying functional 
descriptive material. However, the claim does not define a non-transitory 
computer-readable storage medium or memory and is thus non-statutory for that 
reason (i.e., "When functional descriptive material is recorded on some 
computer-readable medium it becomes structurally and functionally interrelated 
to the medium and will be statutory in most cases since use of technology 
permits the function of the descriptive material to be realized" - Guidelines Annex 
IV). That is, the scope of the presently claimed computer-readable medium can 
range from paper on which the program is written, to a program simply 
contemplated and memorized by a person. In the state of the art, transitory 
signals are commonplace as a medium for transmitting computer instruction and 
thus, in the absence of any evidence to the contrary and give the broadest 
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reasonable interpretation; the scope of a "computer readable storage medium" 
covers a signal per se. In order to overcome the 101 , the "computer readable 
storage medium" should be changed to "non-transitory computer readable 
storage medium". 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

4. Claims 1-4, 7-48, and 50-59 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Subramanian et al. (U.S. Patent 2005/0018775) in view of 
Griffits (U.S. Patent 6,262,776) in view of Markowitz et al. (U.S. Patent 
6,457,052). 

Regarding claim 1, Subramanian et al. discloses a method, 
comprising: determining, by a computing device, whether two or more 
samples of a presentation are processed by a first component of a 
pipeline at an expected time based on a first timing error for a first sample 
and a second timing error for a second sample, the first sample being 
processed before the second sample (paragraph [0049] - the presentation 
time stamp for every B-picture is compared to the system time clock value, 
thereby meaning that two or more samples are processed since every B- 
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picture is sampled and each one has its own result); requesting, by the 
computing device, a second component of the pipeline to alter a manner 
in which the second component processes a portion of the presentation if 
the at least one sample is not processed at the expected time (paragraph 
[0049] - if the PTS value and the STC value differ by more than a 
predetermined threshold, the video decoder drops the B-picture without 
decoding), and taking the appropriate action by the second component in 
response to the requesting, wherein the second component is selected 
from a group comprising multiple components of the pipeline that are 
configured to take the appropriate action in response to the requesting 
(paragraphs [0047]-[0074]). However, Subramanian et al. fails to disclose 
that the second component of the pipeline alters the manner in which the 
second component processes a portion of the presentation of the 
presentation when the two or more samples are not processed at the 
expected time and when the first timing error is greater than the second 
timing error, the altering including taking corrective action to reduce a 
quality of video filtering of the presentation and reducing a quality of audio 
decoding of the presentation, wherein the portion of the presentation 
comprises at least one succeeding sample to the two or more samples; 
and wherein the multiple components is defined to comprise three 
components. 

Referring to the Griffits reference, Griffits discloses a method for 
maintaining synchronization between audio and video comprising: 
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determining whether at least two sample of a presentation is processed by 
a first component of a pipeline at an expected time; and wherein the 
second component of the pipeline alters the manner in which the second 
component processes a portion of the presentation of the presentation 
when the two or more samples are not processed at the expected time 
and when the first timing error is greater than the second timing error, 
wherein the portion of the presentation comprises at least one succeeding 
sample to the two or more samples (Fig. 1 1 - the next sample is altered; 
col. 18, lines 1 1-37 - frames are monitored and once the decoder gets too 
far behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late. 
Furthermore, the first and second samples are being compared in the fact 
that the system is monitoring the process of the decoding and once the 
frames are too early or too late other frames are altered. Griffits discloses 
both situations of the first timing error being greater or less than the 
second timing error since Griffits covers both the fact that frames can be 
too early and too late). 

Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have altered the succeeding 
sample in the presentation as disclosed by Griffits in the method disclosed 
by Subramanian et al. in order to display the samples at the correct times 
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as well as to maintain a synchronization between audio and video data so 
that the viewer can not detect any problems when synchronizing the video 
and audio). However, Subramanian et al. in view of Griffits fails to 
explicitly disclose that the altering includes taking corrective action to 
reduce a quality of video filtering of the presentation and reducing a quality 
of audio decoding of the presentation; and wherein the multiple 
components is defined to comprise three components. 

Referring to the Markowitz et al. reference, Markowitz et al. 
discloses a method for maintaining synchronization between audio and 
video wherein the altering includes taking corrective action to reduce a 
quality of video filtering of the presentation and reducing a quality of audio 
decoding of the presentation (col. 1 , lines 43-46 - by dropping frames the 
quality is reduced). 

Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have reduced the quality of the 
presentation as disclosed by Markowitz et al. in the method disclosed by 
Subramanian et al. in view of Griffits in order to easily synchronize the 
video and audio data. However, Subramanian et al. in view of Griffits in 
view of Markowitz et al. still fail to disclose wherein the multiple 
components is defined to comprise three components. Official Notice is 
taken that the pipeline of multiple components that the video and audio go 
through can comprise of any number of components, wherein the number 
can be any number larger than one. Therefore, it would have been 
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obvious to one of ordinary skill in the art at the time the invention was 
made to have had any one of those components be able to take corrective 
action regarding time management in order to deter incurring a huge lag 
time between components. 

Regarding claim 2, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the first component comprises a media 
sink (Subramanian et al.: paragraph [0049] - media sink - comparing the 
PTS value with the STC value). 

Regarding claim 3, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the second component comprises a 
codec (Subramanian et al.: paragraph [0049] - the decoder drops the B- 
picture therefore, altering the manner in which the decoder processes the 
signal; Griffits: Fig. 2). 

Regarding claim 4, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the first component comprises a media 
sink and the second component comprises a codec (Subramanian et al.: 
paragraph [0049] - media sink - comparing the PTS value with the STC 
value; the decoder drops the B-picture therefore, altering the manner in 
which the decoder processes the signal; Griffits: Fig. 2). 
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Regarding claim 7, Subramanian et al .in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the portion of the presentation comprises 
a frame (Subramanian et al.: paragraph [0049] - one B-picture is a frame; 
Griffits: Fig. 11 - the next sample is altered; col. 18, lines 11 -37 -frames 
are monitored and once the decoder gets too far behind then frames are 
altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the timing of frames are 
monitored; col. 19, lines 1 1-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 8, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that determining whether the two or more 
samples are processed at the expected time comprises comparing a 
timing value in each of the two or more samples to a predetermined time 
frame associated with the presentation (Subramanian et al.: paragraph 
[0049] - the presentation time stamp for every B-picture is compared to the 
system time clock value; Griffits: Fig. 11 - the next sample is altered; col. 
18, lines 11-37 - frames are monitored and once the decoder gets too far 
behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 
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Regarding claim 9, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that determining whether the two or more 
samples are processed at the expected time comprises comparing a 
timing value in each of the two or more samples to a presentation clock 
(Subramanian et al.: paragraph [0049] - the presentation time stamp for 
every B-picture is compared to the system time clock value; Griffits: Fig. 
11 - the next sample is altered; col. 1 8, lines 11 -37 - frames are 
monitored and once the decoder gets too far behind then frames are 
altered in order to allow the decoder to resynchronize the video and audio; 
col. 1 8, lines 39-56 - various statistics on the timing of frames are 
monitored; col. 19, lines 11-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 10, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that determining whether the two or more 
samples are processed at the expected time comprises determining 
whether a respective timing value in the two or more samples was 
processed by the first component at the time specified by the respective 
timing value (Subramanian et al.: paragraph [0049] - the presentation time 
stamp for every B-picture is compared to the system time clock value; 
Griffits: Fig. 11 - the next sample is altered; col. 18, lines 11 -37 -frames 
are monitored and once the decoder gets too far behind then frames are 
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altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the tinning of frames are 
monitored; col. 19, lines 1 1-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 11, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that determining whether the two or more 
samples are processed at the expected time comprises determining 
whether a respective timing value in each of the two or more samples was 
processed by the first component within a given time of a time specified by 
the respective timing value (Subramanian et al.: paragraph [0049] - the 
presentation time stamp for every B-picture is compared to the system 
time clock value; Griffits: Fig. 11 - the next sample is altered; col. 18, lines 
1 1 -37 - frames are monitored and once the decoder gets too far behind 
then frames are altered in order to allow the decoder to resynchronize the 
video and audio; col. 18, lines 39-56 - various statistics on the timing of 
frames are monitored; col. 19, lines 1 1-36 - four situations are determined 
that include frames being too early and too late). 

Regarding claim 12, Subramanian et al. in view of Griffits in view 
of Markowitz et al. discloses all the limitations as previously discussed 
with respect to claim 1 including determining whether the two or more 
samples are processed at the expected time comprises determining if the 
first sample is processed by the first component at a first expected time 
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and determining if the second sample is processed by the first component 
at a second expected time (Subramanian et al.: paragraph [0049] - the 
presentation time stamp for every B-picture is compared to the system 
time clock value and is processed if the PTS value and the STC value 
differ by less than a predetermined threshold; Griff its: Fig. 11 - the next 
sample is altered; col. 18, lines 11 -37 -frames are monitored and once 
the decoder gets too far behind then frames are altered in order to allow 
the decoder to resynchronize the video and audio; col. 18, lines 39-56 - 
various statistics on the timing of frames are monitored; col. 19, lines 11- 
36 - four situations are determined that include frames being too early and 
too late). 

Regarding claim 13, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 , including determining whether the two or more samples 
are processed at the expected time comprises: determining the first timing 
error as a difference between a time at which the first sample is processed 
by the first component and a time at which the first sample is expected to 
be processed; and determining the second timing error as a difference 
between a time at which the second sample is processed by the first 
component and a time at which the first sample is expected to be 
processed (Subramanian et al.: paragraph [0049] - the presentation time 
stamp for every B-picture is compared to the system time clock value and 
is processed if the PTS value and the STC value differ by less than a 
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predetermined threshold; Griffits: Fig. 11 - the next sample is altered; col. 
1 8, lines 1 1 -37 - frames are monitored and once the decoder gets too far 
behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 

Regarding claim 14, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 , including that the first sample comprises an associated 
first timing value and the second sample comprises an associated second 
timing value and wherein determining whether the two or more samples 
are processed at the expected time further comprises determining whether 
the first timing value more closely corresponds to a time at which the first 
sample is processed by the first component than the second timing value 
corresponds to a time at which the second sample is processed by the 
first component (Subramanian et al.: paragraph [0049] - the presentation 
time stamp for every B-picture is compared to the system time clock value 
and is processed if the PTS value and the STC value differ by less than a 
predetermined threshold; Griffits: Fig. 11 - the next sample is altered; col. 
18, lines 1 1-37 - frames are monitored and once the decoder gets too far 
behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
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on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 

Regarding claim 15, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that altering the manner in which the second 
component processes a portion the presentation comprises dropping the 
at least one succeeding sample (Griffits: Fig. 11 - the next sample is 
altered; col. 18, lines 1 1-37 - frames are monitored and once the decoder 
gets too far behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 

Regarding claim 16, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the portion of the presentation comprises 
a plurality of frames, altering the manner in which the second component 
processes the portion of the presentation comprises dropping a subset of 
the plurality of frames, wherein the subset comprises two or more frames 
(Griffits: Fig. 11 - the next sample is altered; col. 1 8, lines 1 1 -37 - frames 
are monitored and once the decoder gets too far behind then frames are 
altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the timing of frames are 
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monitored; col. 19, lines 1 1-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 17, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the first component is a media sink, the 
second component is a codec, and the wherein altering the manner in 
which the second component processes a portion of the presentation 
comprises dropping at least one frame of the presentation (Subramanian 
et al.: paragraph [0049] - the presentation time stamp for every B-picture 
is compared to the system time clock value - media sink; if the PTS value 
and the STC value differ by more than a predetermined threshold, the 
video decoder drops the B-picture without decoding; Griffits: Fig. 11 - the 
next sample is altered; col. 18, lines 11 -37 -frames are monitored and 
once the decoder gets too far behind then frames are altered in order to 
allow the decoder to resynchronize the video and audio; col. 18, lines 39- 
56 - various statistics on the timing of frames are monitored; col. 19, lines 
1 1-36 - four situations are determined that include frames being too early 
and too late). 

Regarding claim 18, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 1 including that the pipeline comprises a media source, a 
media sink, and a topology of media processing nodes; the first 
component is a node in the topology; and the second component is the 
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media sink (Subramanian et al.: paragraph [0049] - the presentation time 
stamp for every B-picture is compared to the system time clock value - 
media sink; if the PTS value and the STC value differ by more than a 
predetermined threshold, the video decoder drops the B-picture without 
decoding). 

Regarding claim 19, Subramanian et al. in view of Griffits in view 
of Markowitz et al. discloses all the limitations as previously discussed 
with respect to claim 1 including that the pipeline comprises a media 
source, a media sink, and a topology of media processing nodes; the first 
component is a node in the topology including a codec; and the second 
component is the media sink (Subramanian et al.: paragraph [0049] - the 
presentation time stamp for every B-picture is compared to the system 
time clock value - media sink; if the PTS value and the STC value differ 
by more than a predetermined threshold, the video decoder drops the B- 
picture without decoding). 

Regarding claim 20, Subramanian et al. discloses a method, 
comprising: determining, by a computing device, when timeliness of 
sample processing in a multi-component pipeline is degrading based on 
processing times of a first sample and a second sample by calculating a 
first timing error for the first sample and a second timing error for the 
second sample (paragraph [0049] - the presentation time stamp for every 
B-picture is compared to the system time clock value; thereby meaning 
that two or more samples are processed since every B-picture is sampled 
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and each one has its own result); and altering, by a component in the 
multi-component pipeline of the computing device, the manner in which a 
component processes a portion of the presentation when the timeliness of 
the sample processing is determined to be degrading (paragraph [0049] - 
if the PTS value and the STC value differ by more than a predetermined 
threshold, the video decoder drops the B-picture without decoding), the 
component is selected from a group comprising multiple components of 
the multi-component pipeline that are configured to take the appropriate 
action in response to a request (paragraphs [0047]-[0074]). However, 
Subramanian et al. fails to disclose comparing the processing times of a 
first sample and a second sample, the altering including taking corrective 
action by reducing a quality of video filtering of the presentation and 
reducing a quality of audio decoding of the presentation, and wherein the 
portion comprises one or more succeeding samples to the first and 
second sample, and wherein the multiple components is defined to 
comprise three components. 

Referring to the Griffits reference, Griffits discloses a method for 
maintaining synchronization between audio and video comprising: 
determining whether at least two sample of a presentation is processed by 
a first component of a pipeline at an expected time; and wherein the 
second component of the pipeline alters the manner in which the second 
component processes a portion of the presentation of the presentation 
when the two or more samples are not processed at the expected time, 
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wherein the portion of the presentation comprises at least one succeeding 
sample to the two or more samples (Fig. 1 1 - the next sample is altered; 
col. 18, lines 1 1-37 - frames are monitored and once the decoder gets too 
far behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late. 
Furthermore, the first and second samples are being compared in the fact 
that the system is monitoring the process of the decoding and once the 
frames are too early or too late other frames are altered. Griffits discloses 
both situations of the first timing error being greater or less than the 
second timing error since Griffits covers both the fact that frames can be 
too early and too late). 

Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have altered the succeeding 
sample in the presentation as disclosed by Griffits in the method disclosed 
by Subramanian et al. in order to display the samples at the correct times 
as well as to maintain a synchronization between audio and video data so 
that the viewer can not detect any problems when synchronizing the video 
and audio). However, Subramanian et al. in view of Griffits fails to 
explicitly disclose that the altering includes taking corrective action by 
reducing a quality of video filtering of the presentation and reducing a 
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quality of audio decoding of the presentation, and wherein the multiple 
components is defined to comprise three components. 

Referring to the Markowitz et al. reference, Markowitz et al. 
discloses a method for maintaining synchronization between audio and 
video wherein the altering includes taking corrective action by reducing a 
quality of video filtering of the presentation and reducing a quality of audio 
decoding of the presentation (col. 1 , lines 43-46 - by dropping frames the 
quality is reduced). 

Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to have reduced the quality of the 
presentation as disclosed by Markowitz et al. in the method disclosed by 
Subramanian et al. in view of Griffits in order to easily synchronize the 
video and audio data. However, Subramanian et al. in view of Griffits in 
view of Markowitz et al. still fail to disclose wherein the multiple 
components is defined to comprise three components. Official Notice is 
taken that the pipeline of multiple components that the video and audio go 
through can comprise of any number of components, wherein the number 
can be any number larger than one. Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was 
made to have had any one of those components be able to take corrective 
action regarding time management in order to deter incurring a huge lag 
time between components. 
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Regarding claim 21, Subramanian etal. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the processing times of the first and the 
second samples are determined relative to a single component in the 
pipeline (Subramanian et al.: paragraph [0049]; Griffits: Fig. 11 - the next 
sample is altered; col. 18, lines 11 -37 -frames are monitored and once 
the decoder gets too far behind then frames are altered in order to allow 
the decoder to resynchronize the video and audio; col. 18, lines 39-56 - 
various statistics on the timing of frames are monitored; col. 19, lines 11- 
36 - four situations are determined that include frames being too early and 
too late). 

Regarding claim 22, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the processing times of the first sample 
is determined relative to a first component in the multi-component pipeline 
and the processing times of the second sample is determined relative to a 
second component in the multi-component pipeline (Subramanian et al.: 
paragraph [0049]; Griffits: Fig. 11 - the next sample is altered; col. 18, 
lines 1 1-37 - frames are monitored and once the decoder gets too far 
behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 
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Regarding claim 23, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the processing times of the first and the 
second samples are determined using timing information in the first and 
second samples (paragraph [0049] - the presentation time stamp for every 
B-picture is compared to the system time clock value; Griffits: Fig. 11 - the 
next sample is altered; col. 18, lines 1 1-37 - frames are monitored and 
once the decoder gets too far behind then frames are altered in order to 
allow the decoder to resynchronize the video and audio; col. 18, lines 39- 
56 - various statistics on the timing of frames are monitored; col. 19, lines 
1 1-36 - four situations are determined that include frames being too early 
and too late). 

Regarding claim 24, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the processing times of the first and the 
second samples are determined using timing information in the first and 
second samples and a presentation clock (paragraph [0049] - the 
presentation time stamp for every B-picture is compared to the system 
time clock value; Griffits: Fig. 11 - the next sample is altered; col. 18, lines 
1 1 -37 - frames are monitored and once the decoder gets too far behind 
then frames are altered in order to allow the decoder to resynchronize the 
video and audio; col. 18, lines 39-56 - various statistics on the timing of 
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frames are monitored; col. 19, lines 1 1-36 - four situations are determined 
that include frames being too early and too late). 

Regarding claim 25, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the timeliness of the sample processing 
is determined based on: a first timing difference between a time specified 
in a timing value in the first sample and a time that the first sample is 
processed by the component in the multi-component pipeline; a second 
timing difference between a time specified by a timing value in the second 
sample and a time that the second sample is processed by the component 
in the multi-component pipeline (Subramanian et al.: paragraph [0049]; 
Griffits: Fig. 11 - the next sample is altered; col. 18, lines 11 -37 -frames 
are monitored and once the decoder gets too far behind then frames are 
altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the timing of frames are 
monitored; col. 19, lines 11-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 26, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the timeliness of the sample processing 
is determined based on: the first timing error being a difference between a 
time specified in a timing value in the first sample and a time that the first 
sample is processed by a first component in the multi-component pipeline; 
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and the second timing error being a difference between a time specified 
by a timing value in the second sample and a time that the second sample 
is processed by a second component in the multi-component pipeline 
(Subramanian et al.: paragraph [0049]; Griffits: Fig. 11 - the next sample 
is altered; col. 18, lines 11 -37 -frames are monitored and once the 
decoder gets too far behind then frames are altered in order to allow the 
decoder to resynchronize the video and audio; col. 18, lines 39-56 - 
various statistics on the timing of frames are monitored; col. 19, lines 11- 
36 - four situations are determined that include frames being too early and 
too late). 

Regarding claim 27, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the timeliness of the sample processing 
is determined by: determining the first timing error being a difference 
between a time specified in a timing value in the first sample and a time 
that the first sample is processed by the component in the multi- 
component pipeline; determining the second timing error being a 
difference between a time specified by a timing value in the second 
sample and a time that the second sample is processed by the component 
in the multi-component pipeline, wherein the second sample is processed 
at a later time than the first sample; and determining that timeliness of 
sample processing is degrading based on the second timing error being 
greater than the first timing error (Subramanian et al.: paragraph [0049]; 
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Griffits: Fig. 11 - the next sample is altered; col. 18, lines 11 -37 -frames 
are monitored and once the decoder gets too far behind then frames are 
altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the timing of frames are 
monitored; col. 1 9, lines 1 1 -36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 28, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the timeliness of the sample processing 
is determined by: determining the first timing error being a difference 
between a time specified in a timing value in the first sample and a time 
that the first sample is processed by a selected component in the multi- 
component pipeline; determining the second timing error being a 
difference between a time specified by a timing value in the second 
sample and a time the second sample is processed by the selected 
component, wherein the second sample is processed at a later time than 
the first sample; and determining that timeliness of sample processing is 
degrading based on the second timing error being greater than the first 
timing error (Subramanian et al.: paragraph [0049]; Griffits: Fig. 1 1 - the 
next sample is altered; col. 18, lines 11 -37 -frames are monitored and 
once the decoder gets too far behind then frames are altered in order to 
allow the decoder to resynchronize the video and audio; col. 18, lines 39- 
56 - various statistics on the timing of frames are monitored; col. 19, lines 
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1 1-36 - four situations are determined that include frames being too early 
and too late). 

Regarding claim 29, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that altering the manner in which the 
component in the multi-component pipeline processes a portion of the 
presentation comprises instructing the component to drop the one or more 
succeeding samples (Griffits: Fig. 11 - the next sample is altered; col. 18, 
lines 1 1-37 - frames are monitored and once the decoder gets too far 
behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 

Regarding claim 30, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that altering the manner in which the 
component in the multi-component pipeline processes the portion of the 
presentation comprises instructing the component to drop each sample in 
a frame of the presentation (Subramanian et al.: paragraph [0049] - if the 
PTS value and the STC value differ by more than a predetermined 
threshold, the video decoder drops the B-picture without decoding; Griffits: 
Fig. 11 - the next sample is altered; col. 18, lines 11-37 - frames are 
monitored and once the decoder gets too far behind then frames are 
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altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the tinning of frames are 
monitored; col. 19, lines 1 1-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 31 , Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that each component comprises processor 
executable instructions executed by a processor (Subramanian et al.: Fig. 
2). 

Regarding claims 32-41, these are apparatus claims corresponding 
to the method claims 1-4, 13-17, and 20. Therefore, claims 32-41 are 
analyzed and rejected as previously discussed with respect to claims 1-4, 
13-17, and 20. 

Regarding claims 42-46, these are computer-readable medium 
claims corresponding to the method claims 1-4, 8, 15, and 20. Therefore, 
claims 42-46 are analyzed and rejected as previously discussed with 
respect to claims 1-4, 8, 15, and 20. Furthermore, Subramanian et al. 
comprises processor executable instructions executed by a processor 
(Fig. 2). 

Regarding claims 47, 48, 50, and 51, these are computer-readable 
medium claims corresponding to the method claims 1-4, 8, 13, and 15. 
Therefore, claims 47, 48, 50, and 51 are analyzed and rejected as 
previously discussed with respect to claims 1-4, 8, 13, and 15. 
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Regarding claim 52, Subramanian et al. in view of Griffits in view 
of Markowitz et al. discloses all the limitations as previously discussed 
with respect to claim 1 including that the portion of the presentation 
comprises a third sample and a fourth sample (Griffits: Fig. 1 1 - the next 
sample is altered; col. 18, lines 11 -37 -frames are monitored and once 
the decoder gets too far behind then frames are altered in order to allow 
the decoder to resynchronize the video and audio; col. 18, lines 39-56 - 
various statistics on the timing of frames are monitored; col. 19, lines 11- 
36 - four situations are determined that include frames being too early and 
too late). 

Regarding claim 53, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claims 1 and 13, including that the method further comprises: if 
the second timing error is greater than the first timing error, taking the 
corrective action, wherein the portion of the presentation comprises two or 
more succeeding samples to the two or more samples (Griffits: Fig. 11 — 
the next sample is altered; col. 18, lines 1 1-37 - frames are monitored and 
once the decoder gets too far behind then frames are altered in order to 
allow the decoder to resynchronize the video and audio; col. 18, lines 39- 
56 - various statistics on the timing of frames are monitored; col. 19, lines 
1 1-36 - four situations are determined that include frames being too early 
and too late). 
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Regarding claim 54, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claims 1,13, and 53 including that the two or more succeeding 
samples are not consecutive samples (Griffits: Fig. 11, step 475 - all B 
frames are dropped regardless of where they are). 

Regarding claim 55, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 20 including that the portion of the presentation comprises 
a third sample and a fourth sample (Griffits: Fig. 1 1 - the next sample is 
altered; col. 18, lines 11-37 -frames are monitored and once the decoder 
gets too far behind then frames are altered in order to allow the decoder to 
resynchronize the video and audio; col. 18, lines 39-56 - various statistics 
on the timing of frames are monitored; col. 19, lines 11-36 - four situations 
are determined that include frames being too early and too late). 

Regarding claim 56, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claims 20 and 28 including that the method further comprises: if 
the timeliness of the sample processing is said determined to be 
degrading, taking the corrective action on the portion of the presentation, 
wherein the portion of the presentation comprises two or more succeeding 
samples to one or more of the first sample and the second sample 
(Griffits: Fig. 11 - the next sample is altered; col. 1 8, lines 1 1 -37 - frames 
are monitored and once the decoder gets too far behind then frames are 
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altered in order to allow the decoder to resynchronize the video and audio; 
col. 18, lines 39-56 - various statistics on the tinning of frames are 
monitored; col. 19, lines 1 1-36 - four situations are determined that 
include frames being too early and too late). 

Regarding claim 57, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claims 20, 28, and 56 including that the two or more succeeding 
samples are not consecutive samples (Griffits: Fig. 11, step 475 - all B 
frames are dropped regardless of where they are). 

Regarding claim 58, Subramanian et al. in view of Griffits in view of 
Markowitz et al. discloses all the limitations as previously discussed with 
respect to claim 42 including that the plurality of components of the 
pipeline are each configured to take corrective action in response to a 
request (Subramanian et al.: paragraphs [0047]-[0074] - the request being 
once there is a timing error detected that component that detects the error 
corrects it). 

Regarding claim 59, grounds for rejecting claim 58 apply for claim 
59 in its entirety. 



Conclusion 

5. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
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See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to HEATHER R. JONES whose telephone 
number is (571 )272-7368. The examiner can normally be reached on Mon. - 
Thurs.: 7:00 am - 4:30 pm, and every other Fri.: 7:00 am - 3:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Thai Tran can be reached on 571-272-7382. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
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PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
91 99 (IN USA OR CANADA) or 571 -272-1 000. 

Heather R Jones 
Examiner 
Art Unit 2621 

HRJ 

July 31, 2010 
/Thai Tran/ 

Supervisory Patent Examiner, Art Unit 2621 



